Self-organized Networks of Competing Boolean Agents 
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A model of Boolean agents competing in a market is presented where each agent bases his action 
on information obtained from a small group of other agents. The agents play a competitive game 
that rewards those in the minority. After a long time interval, the poorest player's strategy is 
changed randomly, and the process is repeated. Eventually the network evolves to a stationary 
but intermittent state where random mutation of the worst strategy can change the behavior of the 
entire network, often causing a switch in the dynamics between attractors of vastly different lengths. 
PACS numbers: 05.65. +b, 87.23. Ge, 87.23.Kg 



Dynamical systems with many elements under mutual 
regulation or influence are thought to underlie much of 
the phenomena associated with complexity. Such sys- 
tems arise naturally in biology, as, for instance, genetic 
regulatory networks jjj, or ecosystems, and in the social 
sciences, in particular the economy Economic agents 
make decisions to buy or sell, adjust prices, and so on 
based on individual strategies which take into account 
the heterogeneous external information each agent has 
available at the time, as well as internal preferences such 
as tolerance for risk. External information may include 
both globally available signals that represent aggregate 
behavior of many agents such as a market index, or spe- 
cific (local) information on what some other identified 
players are doing. In this case each agent has a specified 
set of inputs, which are the actions of other agents, and 
a set of outputs, his own actions, that may be conveyed 
to some other agents. Thus, the economy can be repre- 
sented as a dynamical network of interconnected agents 
sending signals to each other with possible, global feed- 
back to the agents coming from aggregate measures of 
their behavior plus any exogenous forces. 

Each agent's current strategy can be represented as a 
function which specifies a set of outputs for each pos- 
sible input. In the simplest case the agents have only 
one binary choice such as either buying or selling a stock 
||. As indicated first by B. Arthur this simple case al- 
ready presents a number of intriguing problems. In his 
"bar problem" , each agent must decide whether to attend 
a bar or refrain based on the previous aggregate atten- 
dance history 0] . Challet and Zhang made a perspicuous 
adaptation, the so-called minority model, where agents 
in the minority are rewarded, and those in the majority 
punished H. Common to all these and related works 
H is that the network of interconnections between the 
agents is totally ignored. They are mean field descrip- 
tions. Each agent responds only to an aggregate signal, 
e.g. which value (0 or 1) was in the majority for the 
last Ti time steps, rather than any detailed information 
he may have about other specified agents. It is not un- 



expected that an extended system with globally shared 
information can organize. A basic question in studies of 
complexity is how large systems with only local informa- 
tion available to the agents may become complex through 
a self-organized dynamical process. 

Here we explicitly consider the network of intercon- 
nections between agents, and for simplicity exclude all 
other effects. We represent agents in a market as a ran- 
dom network of interconnected Boolean elements under 
mutual influence, the so-called Kauffman network 
The market payoff takes the form of a competitive game. 
The performance of the individual agents is measured by 
counting the number of times each agent is in the ma- 
jority. After a time scale, defining an epoch, the worst 
performer, who was in the majority most often, changes 
his strategy. The Boolean function of that agent is re- 
placed with a new Boolean function chosen at random, 
and the process is repeated indefinitely. Note that it is 
not otherwise indicated to the agents what is rewarded, 
i.e. being in the minority. The agents are only given their 
individual scores and otherwise play blindly; they do not 
know directly that they are rewarded by the outcome of a 
minority game, unlike the original minority game model. 

We observe that irrespective of initial conditions, the 
network ultimately self-organizes into an intermittent 
steady state at a borderline between two dynamical 
phases. This border may correspond to an "edge of 
chaos" §. In some epochs the dynamics of the network 
takes place on a very long attractor; while, otherwise, 
the network is either completely frozen or the dynam- 
ics is localized on some attractor with a smaller period. 
More precisely, numerical simulation results indicate that 
the distribution of attractor lengths in the self-organized 
state is broad, with no apparent cutoff other than the one 
that must be numerically imposed, and consistent with 
power-law behavior for large enough attractor lengths. A 
single agent's change of strategy from one epoch to the 
next can cause the entire network to flip between attrac- 
tors of vastly different lengths. Thus the network can act 
as a switch. 
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Consider a network of N agents where each agent is 
assigned a Boolean variable Ui = or 1. Each agent re- 
ceives input from K other distinct agents chosen at ran- 
dom in the system. The set of inputs for each agent i is 
quenched. The evolution of the system is specified by N 
Boolean functions of K variables, each of the form 



CTj(t + !) = /;( a h (t), a l2 (*),••• u lK (t) 



(1) 



There exists 2 2 possible Boolean functions of K vari- 
ables. Each function is a lookup table which specifies the 
binary output for a given set of binary inputs. In the sim- 
plest case defined by Kauffman, where the networks do 
not organize, each function /; is chosen randomly among 
these 2 2 possible functions with no bias; we refer to this 
case as the random Kauffman network (RKN). 

We will now briefly review some facts about Kauffman 
networks. First, a phase transition occurs on increas- 
ing K . For K < 2 RKN starting from random initial 
conditions reach frozen configurations, while for K > 2 
RKN reach attractors whose length typically grow ex- 
ponentially with N and are called chaotic. RKN with 
K = 2 are critical and the distribution of attractors 
lengths that the system reaches, starting from random 
initial conditions, approaches a power law M, for large 
enough system sizes, when averaged over many network 
realizations. This phase transition in the Kauffman net- 
works can also be observed by biasing the random func- 
tions fi so that the output variables switch more or less 
frequently if the input variables are changed. Boolean 
functions can be characterized by a "homogeneity pa- 
rameter" P which represents the fraction of l's or O's 
in the output, whichever is the majority for that func- 
tion. In general, on increasing P at fixed K, a phase 
transition is observed from chaotic to frozen behavior. 
For K < 2 the unbiased, random value happens to fall 
above the transition in the frozen phase, while for K > 3 
the opposite occurs Q . Kauffman networks are examples 
of strongly disordered systems and have attracted atten- 
tion from physicists over the years (see for example Refs. 
^T|-|l3|). Note that the phase transition previously ob- 
served in Kauffman networks arises by externally tuning 
parameters such as P or K. 

We consider random Boolean networks of K inputs, 
and with lookup tables chosen independently from the 
2 2 possibilities with equal probability. With specified 
initial conditions, generally random, each agent is up- 
dated in parallel according to Eq. 1. The agents are 
competing against each other and at each time step those 
in the minority win. Thus there is a penalty for being 
in the herd. One may ascribe to agents a reluctance to 
change strategies. Only in the face of long-term failure 
will an agent overcome his barrier to change. In the lim- 
iting case of high barriers to change, the time scale for 
changing strategies will be set by the poorest performer 



in the network. The change of strategies is approximated 
as an extremal process Jl4| where the agent who was in 
the majority the most often over a long time scale, the 
epoch, is chosen for "Darwinian" selection. In our simula- 
tions, the network was updated until cither the attractor 
of the dynamics was found, or the length of the attractor 
was found to be larger than some limiting value which 
was typically set at 10,000 time steps, solely for reasons 
of numerical convenience. The performance of the agents 
was then measured over either the attractor or the por- 
tion of the attractor up to the cutoff length. 

The Boolean function of the worst player is re- 
placed with a new Boolean function chosen completely 
at random with equal probability from the 2 2 possi- 
ble Boolean functions. If two or more agents are the 
worst performers, one of them is chosen at random and 
changed. The performance of all the agents is then mea- 
sured in the new epoch, and this process is continued 
indefinitely. Note that the connection matrix of the net- 
work does not evolve; the set of agents who are inputs to 
each agent is fixed by the initial conditions. 
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FIG. 1. Time series of the length of attractor in each 
epoch for K = 3, N = 999 in the stationary state. 

Independent of initial conditions, a K = 3 net- 
work evolves to a statistically stationary but intermit- 
tent state, shown in Fig. 1. Initially the attractors 
that the system reaches are always very long, consistent 
with all previous work on Kauffman networks. But af- 
ter many epochs of selecting the worst strategy, short 
attractors first appear and a new statistically stationary 
state emerges. In this Figure we roughly characterize 
an attractor as "chaotic" or long if its length is greater 
than I = 10, 000 time steps. On varying / a similar pic- 
ture is obtained as long as / is sufficiently large to dis- 
tinguish long period attractors from short period ones. 
In the stationary state, one observes that the network 
can switch behaviors on changing a single strategy. In- 
triguingly, Kauffman initially proposed random Boolean 
networks as simplified models of genetic regulation where 
it is known that switches exist and are important aspect 
of genetic control Jl5| ]. 

To be more precise, the histogram of the distribution 
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of the lengths of the attractor in the self-organized state 
was measured as shown in Fig. 2 for different system sizes 
with the same numerically imposed cutoff I. The appar- 
ent peak at small periods is due to the relative presence 
or absence of prime numbers, and numbers which can be 
factored many ways. The last point represents all attrac- 
tors larger than our numerically imposed cutoff 10,000, 
which is why a bump appears. In between these two re- 
gions, the behavior suggests a power-law, P a tr(t) ~ 1/t 
asymptotically, as is the case at the phase transition in 
RKN H . If we increase or decrease our numerically im- 
posed cutoff then the bump at I correspondingly moves 
left or right and the intermediate region expands or con- 
tracts, both consistent with the power law. Also the 
power law behavior becomes more apparent for increas- 
ing system size suggesting that the self-organized state 
we observed is not merely an effect of finite system size, 
but becomes more distinct as the system size increases. 
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FIG. 2. Histogram of Attractor Lengths for K = 3 Net- 
works. The dashed line has a slope of 1. 

The process of evolution towards the steady-state is 
monitored by measuring the average value of the ho- 
mogeneity parameter P in the network from epoch to 
epoch. As shown in Fig. 3, for K = 3, the av- 
erage value of P tends to increase from the random 
value set by the initial conditions during the tran- 
sient. For finite N, there are fluctuations in P in 
the steady state, as well as finite size effects in the 
average value (P). For N = (99,315,999,3161) we 
measured an average value in the steady state (P) = 
(0.656(1), 0.664(1), 0.669(1), 0.671(1)) and root-mean- 
square fluctuations AP rms ~ (0.015,0.007,0.004,0.001). 
These numerical results suggest that in the thermody- 
namic limit, N — » oo, P is approaching a unique value 
P c ~ 0.672. This value is below the P c ~ 0.792438 §Jl0| 
of random Kauffman K = 3 networks, but is many stan- 
dard deviations away from the initial value. 

The dynamical state that the system evolves toward is 
different from the phase transition of Kauffman networks 
in other (less trivial) ways. In particular, the phase tran- 



sition in RKN is a freezing transition where most ele- 
ments do not change state. Only a few elements, strictly 
(< O(N)), are changing state (lj|^] at the phase tran- 
sition of RKN, whereas in our self-organized networks, 
there can be short attractors associated with many ele- 
ments (~ 0(N)) changing state. This can only occur if 
the Boolean tables in the network become correlated by 
the evolutionary process, which, by construction, is not 
allowed for RKN. Thus our initially chaotic networks are 
not freezing as in Kauffman networks at the phase tran- 
sition, but are somehow phase locking many elements 
together. 
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FIG. 3. Self-organization of the homogeneity parameter 
P for same network as in Fig. 1. The dashed line corresponds 
to the unbiased random value. 

The distribution of performances of agents in the net- 
work fluctuates a great deal from epoch to epoch. The 
performance is measured by counting the fraction of 
times each agent is in the majority. In the case where the 
network has period one, there are obviously two peaks, 
one corresponding to the group always in the minority 
and the other corresponding to the group always in the 
majority. In fact we find that even on the long attractors 
encountered in the steady state, typically a significant 
fraction of the agents are frozen. The number of these 
frozen agents fluctuates from epoch to epoch. 

Fig. 4 is a histogram of performances for agents in a 
self-organized network in a particular epoch which had a 
period greater than 10, 000. Note that the relative per- 
formances vary considerably. The two peaks represent 
the frozen agents. As indicated in the figure, the frozen 
agents are typically divided between the two states un- 
evenly. In any given instant, despite the uneven division 
between the frozen agents, the total number of agents in 
the two states (0,1) is almost evenly divided with fluctua- 
tions that are much smaller than in RKN. Active agents, 
who are changing their state in response to the inputs of 
others, comprise the remainder of the histogram outside 
of the two peaks. As shown in this figure, some agents 
who are inflexible and do not respond to their environ- 
ment perform better than some agents who respond to 
their changing inputs and change states. This suggests 
that somehow the losers are being exploited by some in- 
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formation travelling in the network that they respond 
to. Also, somewhat counterintuitively, a large group of 
agents who take the same action, corresponding to the 
left hand peak, can compete very well in spite of the fact 
that the minority game tends to punish herd behavior. 
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FIG. 4. Histogram of performances in a particular epoch, 
for N = 999 and K = 3 in the self-organized state. Those 
with high scores are poor performers. 

Although we currently have no adequate theoretical 
description of our numerical observations, we can still 
discuss, to some extent, the generality and robustness 
of our results. First, if instead of changing the entire 
Boolean table of the worst performer just one element 
in it is changed, the self-organization process still takes 
place. If on the other hand, the Boolean function of the 
worst performer and those who receive input from it are 
changed, no self-organization takes place. Of course it 
doesn't make sense to change the Boolean functions of 
the agents who listen to the worst performer because in 
our context the barrier to change is an internal function 
of the performance for each individual. The precise be- 
havior on varying K is not determined at present. For 
K — 6, we have simulated systems with N = 99 as long 
as 10 6 epochs and never observed the system to reach 
any frozen state when starting from a random, unbiased 
state in the chaotic phase, so it is possible that the self- 
organization process as described here using completely 
random tables does not occur for high enough K. 

However, other significant modifications were done 
where the self-organization process survives. For exam- 
ple, if instead of changing the boolean tables of the worst 
performer, we keep the boolean tables fixed at their ini- 
tial state, but change the inputs for the worst performer 
by rewiring the network, then the K = 3 networks still 
self-organize to a similar state at an "edge of chaos" with 
similar statistical properties for the periods of the attrac- 
tors and performances of the agents. This occurs despite 
the fact that in this case the average homogeneity pa- 
rameter, P, of the network cannot evolve. 

Rather than define an arbitrary fitness, and select 
those agents with lowest fitness, an approach that was 
used by Bak and Sneppen fL4}| , to describe co-evolution, 



we eliminate the concept of fitness and define a perfor- 
mance based on a specific game. Clearly if the agents are 
rewarded for being in the majority then the behavior of 
the system is completely trivial; the agents gain by coop- 
erating instead of competing and the network is driven 
deep into the frozen phase. This naturally raises the 
question of which types of games lead to self-organized 
complex states. In our model, selection of agents in the 
majority for random change tends to increase the num- 
ber in the minority. Even in the absence of interactions, 
eventually those in the minority would become the ma- 
jority and lose. We suspect that, in general, the game 
must make agents compete for a reward that depends on 
the behavior of other agents in a manner that intrinsically 
frustrates any group of agents from permanently taking 
over and winning. This frustration may be an essential 
feature of the dynamics of many complex systems, and 
our model may be interpreted, as, for instance, describing 
an ecosystem of interacting and competing species. 
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